Sensitivity to noise and ergodicity of an assembly line of cellular automata that classifies density 
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We investigate the sensitivity of the composite cellular automaton of H. Fuks [Phys. Rev. E 55, R2081 (1997)] 
to noise and assess the density classification performance of the resulting probabilistic cellular automaton (PCA) 
numerically. We conclude that the composite PCA performs the density classification task reliably only up to 
very small levels of noise. In particular, it cannot outperform the noisy Gacs-Kurdyumov-Levin automaton, an 
imperfect classifier, for any level of noise. While the original composite CA is nonergodic, analyses of relaxation 
times indicate that its noisy version is an ergodic automaton, with the relaxation times decaying algebraically 
over an extended range of parameters with an exponent very close (possibly equal) to the mean-field value. 

PACS numbers: 05.50.+q, 89.75.Fb, 89.20.Ff 

Keywords: Density classification task, probabilistic cellular automata, Gacs-Kurdyumov-Levin, robustness, ergodicity, noise 



I. INTRODUCTION 

The density (or majority) classification task for one- 
dimensional two-state cellular automata (CA) is a well known 
problem in theoretical computer science and, more generally, 
in the theory of complex systems [1—3]. The task consists of 
classifying binary strings according to their density p of ones 
using local rules only, and it is completed successfully if a 
correct verdict as to whether p < 1/2 or p > 1/2 is obtained 
in finite time, at most linear in the size L of the input string. 

The density classification task originated in a paper by 
Gacs, Kurdyumov, and Levin (GKL) that was primarily con- 
cerned with providing evidence for the existence of local rules 
stabilizing a given phase against noise despite the ergodicity 
of the model [4, 5]. For locally interacting systems of au- 
tomata, the density classification task is a nontrivial task, be- 
cause the cells have to achieve a global consensus cooperating 
locally. Systems with a controlling unit or layer such as a com- 
puter with a memory controller or a hierarchical neural net- 
work can overcome this difficulty in O(L) or even 0(1) time 
by peeking each cell and accumulating its contents on a sep- 
arate register. For one-dimensional systems of autonomous 
and memoryless cells interacting locally this is not an option; 
emergence of collective behavior is required in these cases. 

It has been argued that the density classification task can- 
not be achieved without misclassifications by a single locally 
interacting two-state cellular automaton in any dimension [6]. 
Indeed, under the requirement that all the cells of the automa- 
ton must converge to the same state as the majority state in 
the initial configuration, no automata conceived to date could 
achieve 100% efficiency. (For a critique on this classification 
criterion see [7].) The GKL automaton achieves a good 81.6% 
performance in a sort of standard test consisting of classify- 
ing something between 10 4 and 10 7 random initial configu- 
rations of an L = 149 array close to the "critical" density 
p = 1/2. Improvements by humans over the GKL rule could 
achieve 82.2% of correct classifications, and late genetic and 
co-evolution programming techniques were able to upgrade 
the success rate to 86.0% [8-13]. 



The no-go results mentioned above, however, did not rule 
out more complicated CA. In 1997, H. Fuks introduced a com- 
bination of CA that solves the density classification problem 
perfectly, with 100% efficiency [14]. The composite CA of 
Fuks corresponds to an "assembly line" of two CA (rules 184 
and 232 in Wolfram's enumeration scheme; cf. Sec. II), with 
one CA (184) running for the first half of the dynamics and 
the other CA (232) running for the second half. The time T 
needed to complete the task is bounded by T L — 2. More- 
over, when L is even the assembly line CA deals with the case 
p = 1/2 properly, by converging the initial configuration to 
the final configuration • • ■ 010101 ■ ■ •. Generalizations to arbi- 
trary rational densities and to n-ary CA followed suit [15]. 

In this article we investigate the classification performance 
of Fuks's assembly line CA under the influence of noise, turn- 
ing the CA into a probabilistic cellular automaton (PCA) in 
which the transitions have a small probability of erring the fi- 
nal state. The classification efficiency of the PCA is assessed 
under several levels of noise in each of its components and 
for different system sizes, including even ones, by means of 
numerical experiments. We also compare the performance of 
the PCA with that of the noisy GKL automaton and provide 
evidence for its ergodicity at any level of noise. 

This article is organized as follows. In Sec. II we intro- 
duce the assembly line CA together with its PCA version and 
describe our numerical experiments. In Sec. Ill we present 
our results and discuss their implications for the reliability of 
the PCA, including a discussion of its ergodicity. Finally, in 
Sec. IV we summarize our findings, make a few remarks and 
set forth some perspectives for future developments. 



II. THE ASSEMBLY LINE CA AND ITS PCA VERSION 

A two-state cellular automaton transition function $ on a 
one-dimensional array of L cells is a map that evolves a given 
configuration T] = (rji, 772, • • • , tjl) G {0, 1} L in discrete time 
t £ N according to 

V (t + 1) = *(f,(t)) = (*ifa(t)), • • ■ , (D 
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with each ^i(rj(t)) = $i (%_{,(*), . . . , Tfr(t), . . .,r] i+ri (t)) a 
function of {0, l}^^ 1 -> {0, 1}. 



2 



TABLE I. Rule tables for the composite assembly line CA $232- For 
each CA, the first row lists the initial neighborhood and the second 
row lists the state reached by the central bit of the initial neighbor- 
hood under the action of the CA rules. 



CA184: 111 110 101 100 Oil 010 001 000 
10 1110 



TABLE H. Rule tables for PCA $>\H{e' ,e"). Reads like Table I, 
except that the central bit of the initial neighborhood reaches its final 
state with the probability given at the leftmost column. 



PCA 184: 111 110 101 100 011 010 001 000 
1 - e' 10 1110 
e' 1 1 1 1 



CA232: 111 110 101 100 011 010 001 000 



1 1 



1 
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A convenient way to express the local rules $i when they 
are all equal is by means of a rule table. For the assembly 
line of Fuks, we have two rule tables, one for each part of the 
assembly line. These rule tables are given in Table I. We refer 
to this composite rule as CA $232- The numbers 184 and 232 
come from reading the pattern of final states in the second row 
of each rule table as a binary number, after S. Wolfram [1]. 
Since $232 i s a composite CA, we must specify which part 
runs when and for how long. By definition, in this assembly 
line the CA 184 part runs for the first Ti 84 = [(L- 2) /2J time 
steps and the CA 232 part runs for the last T 232 = [(L- l)/2j 
time steps [14]. Under these running times, $232 performs the 
density classification task perfectly and independently of L. 

We are interested in the noisy version of $232' which can 
be obtained by allowing errors to intervene in the transitions 
made by $232- We thus introduce error rates e' and e", ^ 



e'e" € 



respectively to the CA 184 and CA 232 parts of 
the dynamics, turning CA $232 mto p CA (e', e"). Under 
this PCA, each cell rji(t) takes the value rji(t+ 1) that it would 
take under the noiseless $232 dynamics with probability 1 — e 
or its complement 1 — rji(t + 1) with probability e, where e 
equals e' or e" depending on which CA (184 or 232) is ruling 
the dynamics at instant t. In what follows, when the context 
is clear we sometimes refer to an unprimed e to denote either 
e' or e". The rule tables for this PCA are given in Table II. 

The performance of a CA or PCA in the density classifi- 
cation task can be accounted as the fraction of initial config- 
urations r)(0) that, beginning with density ,0(77(0)) = x end 
up after T time steps as a configuration i~i(T) with density 
p(r](T)) equal to 0, 1/2 or 1 depending on whether the initial 
density was a; < 1/2, x = 1/2, or x > 1/2. For <S>lU(e' , e"), 
we fix T = Tig4 + T232 and define the efficiencies as 
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E ( i\e' 1 e") = {E^e',e"){x>\/2)), (3b) 



E*(e',e") = {E*{e', s"){x)), (3c) 

together with E^\e',e") = (E^(e',e")(x = 1/2)) when L 
is even. Notice that the overall efficiency -E$(e', e") is a sim- 
ple average over the performance of the automaton at all den- 
sities probed. It cannot be compared with the efficiencies ob- 
tained using initial configurations sampled close to some spe- 
cific density or distributed according to some specific proba- 
bility distribution other than the flat one over ^ p ^ 1. 

In this article we investigate the classification performance 
of $232 (e' , e") for one-dimensional arrays of cells under pe- 
riodic boundary conditions. We analyze systems of several 
lengths, both odd and even, because contrary to the GKL 
and related CA, the behavior of $232 ls we U defined when 
L is even and we want to see whether and by how much 
$232 £ ") inherits the performance of the noiseless version 
at p = 1/2. In our simulations, we pick 1000 sample configu- 
rations for each value of p chosen and average the classifica- 
tion performance of $232 £ ") f° r eacn sampled configura- 
tion over 1000 realizations of the noise. Noise is thus dynamic 
in our simulations, not quenched. A typical efficiency profile 
with fixed L, e', and e" is traced out of data for ^100 differ- 
ent values of p. This amount of sampling suffices to provide 
smooth curves and useful figures. The uncertainties in the fig- 
ures are mostly of the order of the sizes of the symbols used 
to display them. The reader may assume that the errors are 
typically of the order of ±2% of the main figure given. The 
results of our experiments are presented in the next section. 



Ez(e',e")(x) = 



' #{r l {T):p{r } {T)) = e{x-l/2)} 
#{77(0) : p( V (0)) = x) 



(2) 

where is the Heaviside step function and the angle brack- 
ets denote average over the configuration space as well as over 
realizations of the noise. Notice that the case of L even and 
x = 1/2 is automatically accounted for the efficiency of the 
CA or PCA, in agreement with the behavior of the noiseless 
$232- It i s a l so useful to define the average efficiencies 



E ( £\e',e") = {Es > {e l ,e"){x<\/2)), 



(3a) 



III. RESULTS OF THE NUMERICAL EXPERIMENTS 



A. The efficiency of the PCA 



We begin by considering the case of an even number of 
cells, L = 150, just to verify how E^ 2 behaves. The efficien- 
cies of $232 ( e '> e ") m this case subject to noise in the interval 
0.0005 < e',e" sC 0.015 along the lines e" = and e' = 
appears in Fig. 1. Total efficiency profiles for several different 
levels of noise appear in Fig. 2. 
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FIG. 1. Efficiencies of $232 (e', e") along the lines (a) e" = and 
(6) e' = in a PCA of L = 150 cells. Notice how e£\ e£\ 
and E® superimpose greatly and cannot be discerned clearly in the 

graphs. This results from the symmetry E^' o E^' about p =1/2 

( 1 ) 

and the steep decay of E^ 2 toward zero with increasing noise. 

The data reveal that $232 £ ") is highly sensitive to noise. 
From Fig. 1, it is clear that already at very small levels of 
noise its classification performance becomes significantly de- 
graded and barely compares with the average performance 
of the noiseless GKL automaton (namely, ~81.6% around 
p = 1/2 and ^97.7% overall), an imperfect classifier. This 
observation posed us to compute the solutions of the equation 

E*(e)(p) =E GKL (v)(p) (4) 

along the line e' = e", where J?gkl( j/ )(p) is the efficiency 
of the GKL automaton under noise ^ v ^ 1/2 (cf. Ap- 
pendix for the definition of this automaton). In order to solve 
the above equation, we fix p and e and find the v that sat- 
isfies it. In this comparison test, we ran both automata with 
L = 149 cells and allowed only T = L time steps for the 
GKL automaton to classify the densities, when the custom- 
ary is to run it through T — 2L time steps. In principle this 
grants some advantage to $232 ( e ' j £ ")' hut we found it neg- 
ligible. A few loci of Eq. (4) for different choices of p are 
shown in Fig. 3. We invariably found v > e, meaning that 
^GKi^W) is more robust and classifies density more reliably 
that $232 ( £ 'i e ") under the influence of noise, whether over 
the line e' = e" or along nearby paths. 

Noise is particularly deleterious to the performance of 
$232( e 'i e ") close to p = 1/2, a region where < &gkl(i / ) 
proved far more robust. From Fig. 3 we see that along most 
of the curve for p = 70/149, ^gklM can cope with as 
much as eight times more noise than $232 ( e ' \ £ ") f° r the same 
performance, and the trend seems to indicate that this factor 
can become even greater at higher levels of noise or closer to 
p = 1/2. Notice, however, that increasing levels of noise 
"disrupt" the PCA at some point, above which £$ — > 0. 
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FIG. 2. Overall efficiencies E<f,(p) for $232 (e'> e") along the lines 
(a) e" = and (6) e' = in a PCA of L = 150 cells, as in Fig. 1. 
From the uppermost curve downward, e' or e" equals 0.0005, 0.001, 
0.003, 0.005, 0.007, and 0.01. 

In an automaton of L = 149 cells, this point of rupture for 
$ 232(e'^") alon S the line e 1 = e" is close to e ~ 0.0152 
and for $gkl(^) it is close to v ~ 0.053. Notice that the re- 
sults reported in [5] regarding the ergodicity of the noisy GKL 
automaton were obtained mostly beyond its point of rupture. 

Figures 1 and 2 reveal that $232 £ ") is slightly more sen- 
sitive to noise in its CA 232 second half than in its CA 184 first 
half. Otherwise, for initial densities far off the critical region 
around p = 1/2 the C A 232 part is more robust to noise, in the 
sense that when the noisy CA 232 part receives a configuration 
preprocessed by a noiseless CA 184 part it misses the correct 
classification less than when it receives a configuration pre- 
processed by a noisy CA 184 with the same level of noise and 
processes it without noise. The case most sensitive to noise is 

p = 1/2, as it can be verified from the behavior of E^ 2 . This 
is an expected treat, since initial states with p = 1/2 are the 
ones for which the smallest error favoring either side of the 
density can most probably lead to an erroneous classification. 

Put together, Figs. 1 and 2 allo w us to conclude that except 
for very small error rates, say, \/ e'e" < 0.001, and for input 
strings with a very definite density, say \p — 1/2 1 > 1/3, PCA 
$232( £ 'j £ ") is an unreliable density classifier. Figure 3 also 
tells us that the GKL automaton is a better option for classify- 
ing density in noisy environments or using faulty components. 

B. The ergodicity of the PCA 

Clearly, $^(e' = 0,e" = 0) is nonergodic. When e 1 ^ 
or e" 7^ or both, however, we can ask whether the PCA 
sweeps through the configuration space or gets clogged in 
some finite neighborhood, signaling nonergodicity. We then 
define r$(L,e',e") as the time it takes the initial configu- 
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FIG. 3. Loci of "isoefficiency" between $232 (s) an d $gkl(i / ), 
Eq. (4), for a few choices of p (indicated as fractions). Notice the 
different ranges of the axes. For fixed p, $232 (e) performs worse 
than $GKL(f) for any noise levels (e, z/) beneath the respective iso- 
efficiency curve. The line v = e (dashed) is shown for reference. 



ration of all l's (not absorbing if e' or e" or both are non- 
null) to evolve into a configuration with density p < 1/2. 
We take p = 1/2 as the threshold because when the system 
reaches a configuration with this density it has, in an intuitive 
sense, "crossed the barrier to the other side of the well." For 
^232 ( e 'i there is the problem of defining the shares of 
each CA of the assembly line in this relaxation time. Here we 
take T 184 = T232 over increasing time spans. Beginning with 
some small Xig4 = T232, we evolve the initial configuration 
through T = Tisa + J232 time steps, measure the density and, 
if p ^ 1/2, increment T 18i <- T 1S4 + 1, T 232 <- T 232 + 1, 
run the PCA with an initial configuration of all l's through 
the updated T — Ti$4 + T232 and measure p again, and repeat 
this procedure until p < 1/2 for some T. We then estimate 
r$(L, e' , e") from an average over 1000 such hitting times. 

We first estimated t$(L, e' , e") along the line e' = e". Fol- 
lowing [5], we look for flipping times of the form 

r$(L,e) ~ exp(/(L,e)), (5) 

possibly with an algebraic prefactor. The putative relation 
(5) is drawn on an analogy between the space-time diagram 
of the PCA and the configuration of a 2D interacting clas- 
sical spin-i model over A = {(/, t) £ {1, 2, . . . , L} X N}. 
In this analog 2D spin model, close to the critical point 
< \T - T c \/T c < 1 the correlation length in the t di- 
rection scales like £\\(L,T) ~ \T - T c \~ v \\ and also like 
£||(L,T) - exp(L z /T) for T < T c and £\\(L,T) ~ L z at 
T = T c , being bounded in L for T > T c [16, 17]. The real 
numbers and z are critical exponents. The 2D spin model 
scenario is translated into the PCA scenario by identifying e 
with T and r$(L,e) with £m(Z,T), with the ergodic phase 
of the PCA corresponding to the disordered (T > T c ) phase 
of the spin model. A nonergodic dynamics would thus imply 
that r$(L, e) diverges as L j" 00 with finite e's, while for an 
ergodic dynamics f(L,e) should remain bounded in L. Obvi- 
ously, r$(L, e) is expected to diverge as e 4,0. 

Plots of lnr$(i, e) for fixed odd values of L and e appear 
in Figs. 4 and 5. The errors in the data are in the range of 
5-10%, with higher uncertainties for smaller L and larger e. 
We found that for fixed L, In r$(L,e) varies with lne. Both 
the conditions that r$(L, e) diverges as e 4-0 and that f(L, e) 
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FIG. 4. Relaxation times t$(I/, e) for some fixed L between L = 
149 (lowermost curve) and L — 3499 (uppermost curve); cf. Fig. 5. 
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FIG. 5. Relaxation times t<s>(L,e) for some fixed e between e = 
0.0001 (uppermost curve) and e = 0.03 (lowermost curve). 



remains bounded in L are observed by the data. Moreover, 
from Fig. 4 we already see that f(L, e) is quite insensitive to 
L — actually, the curves clump together more as L gets larger, 
indicating sublinear growth of f(L,e) with L. Linear fits to 
the curves give slopes 0.91 ^ oil ^ 0.97, all with correlation 
coefficients R 2 > 0.9998. An extrapolation of the regression 
coefficients qj, x L^ 1 gives = 0.957 ± 0.004 with a cor- 
relation coefficient R 2 ~ 0.89. This provides considerable 
evidence for the ergodicity of $232 ( £ 'i at l east on the line 
0.0001 ^ e' = e" ^ 0.03. Complementary evidence comes 
from Fig. 5, where we see that, for fixed e, the relaxation times 
are largely independent of L. The interpretation of this behav- 
ior is that no matter how far the initial configuration 11 ■ - ■ 1 is 
from the typical stationary configurations, the system is able 
to reach them in finite (and, actually, relatively short) times, 
forgetting its initial configuration. In all cases we found that 
r$(i,e',e") ~ e~ a (with e = e' or e" — it does not really 
matter) and exponent in the range 0.91 ^ a ^ 0.98. 

Altogether, these facts indicate that $232 ( £ ' 1 £ ") i s probably 
ergodic for nonzero levels of noise. 

We would like to remark that the exponents we found for 
r$(L, e', e") are all very close to the mean-field value ^ = 1 
expected for a completely uncorrected behavior. The fact that 
the PCA is very sensitive to noise is an indication that its sta- 
tionary states in the ergodic region are very uncorrelated in 
space and time, and we guess that its critical behavior in the 
disordered phase is probably ruled by mean-field exponents. 
We repute the deviation of the values of a observed in our 
simulations from the exact mean-field value not only to sta- 
tistical errors but also to finite-size effects — indeed, we found 
larger qj, for larger L and e. 
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IV. SUMMARY AND FURTHER DEVELOPMENTS 

We conclude that while $232 can classify density perfectly, 
its noisy version is highly sensitive to noise, almost to the 
point of being useless. Since error detection and correction 
are not available to locally interacting CA or PC A, any real, 
noisy- world application of $232 nas to ponder its sensitivity to 
noise and the ensuing limitations. We have also accumulated 
evidence for the ergodicity of $232 £ ") at an Y finite level of 
noise, meaning that the mechanism that drives the CA into its 
absorbing configurations does not survive to noise. The criti- 
cal behavior of the PCA close to and above the critical point 
(e' , e") c = (0, 0) is probably ruled by mean-field exponents. 

It would be interesting to investigate the robustness of 
$232( £ 'i e ") m more general graphs. Preliminary results in- 
dicate that single CA rules do not perform well under such 
conditions [18, 19], but these results do not extend prima fa- 
cie to composite CA like $232 ■ A study of the efficiency of 
the GKL rule under the influence of both noise and graph 
topology was carried out in [20], with the conclusion that the 
noisy GKL rule performs worse than a simple majority rule in 
small-world networks. Results obtained for random Boolean 
networks also point to a better performance for these models 
than for CA or PCA [21]. Notice that while the CA 232 part 
of $232 i s me majority rule that can be easily extended to any 
node topology, the C A 1 84 part would have to be redesigned 
to work with nodes of degree k > 2. Analytical approaches to 



$232( £, 7 e ") are a l so desirable, although rigorous results for 
PCA are hard to obtain. Mean-field approximations, however, 
are amenable to analysis and may help to set forth new results 
for this class of PCA [22]. 
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Appendix: The noisy GKL automaton 

In Sec. Ill A we were led to investigate the noisy GKL au- 
tomaton. For the sake of completeness, we specify it here. 

The noiseless GKL automaton evolves according to the 
following rules [4]: if r)i(t) — 0, then rji(t + 1) = 
*gkl(%(*)) = 9(-m- 3 (t) + Vi-i{t) + m(t) - 3/2), where 
#(■) is the Heaviside step function introduced in Eq. (2), and 
if r)i{t) = 1, then Vi (t + 1) = $GK L (^(i)) = 0(TH(t) + 
+ 771+3 (t) — 3/2) — i.e., r)i(t + 1) equals the majority 
of its particular neighborhood at instant t, that itself depends 
whether rji{t) = or 1. The noisy version just adds the possi- 
bility that, at every site, instead of r)i(t + 1) = &GKh{Vi(t))> 
with probability v we have r)i(t + 1) = 1 — &GKh(Vi(t)) [5]- 
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